The present invention relates to a technology which detects, in an electronic camera equipped with a solid image pickup element, a defective pixel of the solid image pickup element, and corrects output signals from the defective pixel.
A solid image pickup element equipped in an electronic camera has a function to convert, with many pixels arranged two-dimensionally, optical images of a subject formed on the pixel into an amount of electric charges (electric signals), and to output them. Among these pixels, incidentally, there can be some pixels which can not output normal signals due to the defects (pixel defects) based on dust adhesion and crystal defects. These pixel defects are represented by both a white flaw wherein signals in which excessive signal components are added to output signals which are supposed to be outputted in accordance with luminance of a subject are outputted accidentally to make an image to be whitish, and a black flaw wherein signals in which certain signal components are subtracted from output signals which are supposed to be outputted in accordance with luminance of a subject are outputted accidentally to make an image to be blackish.
When there are caused much pixel defects, it is apprehended that image quality is deteriorated badly, when reproducing images which are picked up by the use of the solid image pickup element having that pixel defects. On the other hand, with regard to the solid image pickup element having therein at least hundreds of thousands of pixels or more which has recently come to be used, it is actually difficult to manufacture the solid image pickup element which is free from pixel defects. It is therefore required to use the solid image pickup element on the assumption that pixel defects to a certain extent always exist.
There has already been developed an electronic camera which is equipped with a correction circuit which corrects, through post-processing, the electric signals outputted from the pixel having pixel defects to improve image quality, based on the assumption mentioned above. The electronic camera stated above employs a method wherein a pixel having pixel defects (defect pixel) in the solid image pickup element is detected by the use of a pixel defect inspecting instrument on the occasion of factory shipments of the electronic camera, and a position thereof is stored in a ROM accompanying the electronic camera, for example, as information, whereby signals outputted from the defect pixel are corrected properly in the case of actual photographing.
However, it was found that the white flaw stated above is sometimes increased in accordance with aging deterioration or environmental conditions where the solid image pickup element is used, because it is a pixel defect based on a crystal defect. For example, the white flaw tends to be increased when the temperature surrounding the solid image pickup element is raised, or when the exposure time is long. In such a case, another defect pixel is caused at a position which is different from the position of the defect pixel detected under a prescribed condition on the occasion of factory shipments, which sometimes makes it impossible to correct sufficiently. Therefore, it is also possible to consider a method wherein exposure for a long time or exposure at high temperature is conducted to obtain defect pixel information, and signals from the defect pixel are always corrected based on the aforesaid information.
However, the white flaw caused by the exposure for a long time is one which is not caused by exposure for a short time, and it is not preferable to conduct correction processing at all times with regard to the white flaw, because the correction processing delays the processing time. Further, since the pixel causing a white flaw under exposure for a long time can function as a normal pixel in the course of ordinary photographing, it is apprehended that correction for the normal output signals from that pixel may rather deteriorate image quality.
In particular, a digital camera is in a tendency that the number of pixels of a solid image pickup element is increased for improvement of image quality, and this tendency of increase of the number of pixels has created a tendency that the number of defect pixels is naturally increased and the number of occurrence of another white flaw caused by aging change is increased in proportion to the increase of the number of pixels. Therefore, in a digital camera having many pixels of the solid image pickup element, the defect pixel stated above matters much especially, and a technology to cope with aging change of the defect pixel in the market place has been desired.
The invention has been achieved in view of the problems stated above, and its object is to provide an electronic camera which can cope with aging changees of defective pixels of a solid image pickup element and can detect the defect pixels accurately. Further object of the invention is to provide an electronic camera which can properly correct white flaws the number of which depends on a length of time of exposure, a pixel signal correcting method and a recording medium.
The objects mentioned above can be attained by the structures or methods described in each item below.
Item 1
An electronic camera equipped with a solid image pickup element wherein an image for detecting pixel defects is subjected to image pickup, and image data of each pixel of the solid image pickup element obtained through the image pickup is compared with reference image data respectively to detect defect pixels in the solid image pickup element.
In the structure mentioned above, an image suitable for detecting pixel defects is photographed for image pickup by an electronic camera, and image data of each pixel of the solid image pickup element obtained through the image pickup are compared with image data representing a reference to detect defect pixels for each image pickup of the image for detecting pixel defects.
Item 2
The image data representing a reference is assumed to be the value stored in advance.
In the structure, the reference image data for judging whether image data are normal or abnormal in accordance with the image for detecting pixel defects are stored in advance, and image data obtained through image pickup of the image for detecting pixel defects are compared with the reference image data for each pixel, whereby the image data are judged whether they are normal or abnormal for each pixel.
Item 3
The reference image data are operated based on image data in a prescribed area which includes a targeted pixel.
In the aforesaid structure, a defect pixel is detected by judging whether or not the targeted pixel is outputting image data which are on the same level as in the normal pixel, with the image data around the targeted pixel which are assumed to be image data obtained by the normal pixel.
Item 4
Color filters respectively for plural different colors are provided on the solid image pickup element, and the reference image data stated above are represented by a mean value of image data of the pixel on which a color filter having the same color as in the targeted pixel in the prescribed area is provided.
According to the aforesaid structure, in an electronic camera having the structure wherein each pixel is provided with each of color filters respectively for R, G and B and image pickup for a color image is conducted, a mean value of image data of the pixel which is located around the targeted pixel for judging the defect pixel and is provided with a filter having the same color as in the targeted pixel is calculated as image data in the normal pixel, and this calculated value is compared with image data of the targeted pixel to judge whether the image data of the targeted pixel are normal or abnormal.
Item 5
A prescribed area including the targeted pixel is assumed to be a prescribed area in which the targeted pixel is centered.
In the structure stated above, a mean value of image data of the pixel which is provided with a filter having the same color as in the targeted pixel is obtained in the image area in which the targeted pixel is centered for each targeted pixel, and this obtained value is regarded as a normal value to detect defect pixels.
Item 6
Image data obtained through image pickup of the image for detecting pixel defects are split into plural blocks, and each split block is made to be a prescribed area including the targeted pixel.
In the structure stated above, results of image pickup are split into plural blocks with a pixels x b pixels serving as a unit, for example, then, a mean value is obtained for each same filter color for each block, and this mean value is compared with the image data of the targeted pixel. Therefore, with regard to the targeted pixel in the same block, detection of defect pixels is conducted by the use of the same mean value.
Item 7
From the mean value stated above and image data of the targeted pixel, contrast of the targeted pixel is obtained, and the contrast is compared with the prescribed threshold value to detect a defect pixel.
In the structure stated above, image data of the targeted pixel are judged whether they are normal or abnormal based on the level of the contrast for pixels located around the targeted pixel and thereby, the targeted pixel is judged whether it is a defective pixel or not.
Item 8
A difference between the mean value and image data of the targeted pixel is obtained, and the difference is compared with the prescribed threshold value to detect a defective pixel.
In the structure stated above, when the difference between image data of the targeted pixel and average image data of surrounding pixels is great, the image data of the targeted pixel are judged to be abnormal, and thereby the targeted pixel is detected to be a defective pixel.
Item 9
Image data obtained through image pickup of the image for detecting pixel defects are divided into plural blocks, then a mean value of image data is obtained for each color of color filters in the divided block, and a contrast between image data of the targeted pixel and the aforesaid mean value for the same color obtained in the block containing therein the targeted pixel is obtained, and then the contrast is compared with the prescribed threshold value to detect a suspected defective pixel. While in the prescribed area wherein the suspected defective pixel thus detected is centered, a mean value of image data of a pixel provided with a color filter which is the same as that for the aforesaid suspected pixel, and a difference between the mean value and the image data of the targeted pixel is obtained to compare the difference with the prescribed threshold value for detecting a defective pixel.
In the structure stated above, the pixel area is first divided into plural blocks, and a mean value of image data corresponding to the same color filter is obtained for each block. Then a contrast between the mean value and image data of the targeted pixel is calculated, and a suspected defective pixel is detected based on the contrast. Next, in the prescribed area wherein the defective pixel detected as a suspected one is centered, a mean value of pixels of the same color is obtained, and a difference between the mean value and image data of the suspected pixel is calculated to judge whether the pixel detected as the suspected one is a defective pixel or not, based on the difference.
In the same block, in this case, the same mean value is used to judge whether the targeted pixel is a defective pixel or not. Therefore, it is not necessary to obtain a mean value of peripheral pixels for each targeted pixel in detection of a suspected defective pixel. Further, calculation of a mean value in an area where the targeted pixel is centered is limited to the pixel which has been detected as a suspected defective pixel.
Item 10
In the course of image pickup of the image for detecting the pixel defects, there is conducted defocus control which deviates a focus forcibly from the focused position.
In the structure mentioned above, fine flaws or shadows on an image for detecting the pixel defects are vignetted by the defocus control.
Item 11
A diffusing plate which diffuses incident-light in the solid image pickup element is provided to conduct image pickup of the image for detecting the pixel defects.
In the structure mentioned above, light is diffused by the diffusing plate, and the solid image pickup element is illuminated uniformly with light.
Item 12
When conducting image pickup of the image for detecting pixel defects, an aperture and/or shutter speed is adjusted so that luminance of the image picked up may be within a prescribed range.
In the structure stated above, image data wherein a contrast for a black flaw is sufficiently great can be obtained even when luminance of the image for detecting pixel defects is low, and when luminance of the image for detecting pixel defects is too high, in contrast with the foregoing, image data wherein a contrast for a white flaw can sufficiently be secured can be obtained.
Item 13
The image for detecting pixel defects is subjected to image pickup plural times, and a defective pixel is detected from the results of plural implementation of image pickup.
In the structure stated above, a defective pixel is not specified only from the results of one implementation of image pickup, for example, but the final defective pixel (including a suspected defective pixel) is detected when the defective pixel is detected from the results of prescribed number or more of implementation of image pickup, for example, among results of plural number of implementation of image pickup.
Item 14
The image for detecting pixel defects is subjected to image pickup plural times, and a defective pixel is detected based on images wherein the images obtained from image pickup for plural times are superposed.
In the structure stated above, the image for detecting pixel defects is subjected to image pickup plural times, and image data obtained from the foregoing are added for each pixel, and a defective pixel is detected based on the image data of the results of the addition.
Item 15
Incident-light into the solid image pickup element is intercepted, and a dark image in this case is subjected to image pickup as an image for detecting pixel defects.
In the structure stated above, Incident-light into the solid image pickup element is intercepted through adjustment of an aperture, and a dark image under this condition is subjected to image pickup, and detection of a white flaw is conducted in a way wherein image data of normal pixels are made to be on the same level as that of the minimum luminance level.
Item 16
The image for detecting pixel defects is subjected to image pickup when turning on a power supply switch, and whereby a defective pixel is detected.
In the structure stated above, an image for detecting pixel defects (for example, a dark image) is automatically subjected to image pickup for each turning on of a power supply switch, so that a defective pixel is detected.
Item 17
A temperature of the solid image pickup element is detected, and the image for detecting pixel defects is subjected to image pickup when the detected temperature is not lower than the prescribed temperature so that a defective pixel is detected.
In the structure stated above, temperature rise of the solid image pickup element which causes a change in increase of white flaws is detected, and when the temperature rise is observed the image for detecting pixel defects is subjected to image pickup so that a defective pixel is detected.
Item 18
The image for detecting pixel defects is subjected to image pickup at a prescribed interval so that a defective pixel is detected.
In the structure stated above, the image for detecting pixel defects is subjected to image pickup at a prescribed interval, such as, monthly, semiannually, or yearly, for example, so that a defective pixel is detected.
Item 19
A position of the defective pixel detected is stored, and image data of the defective pixel are corrected based on information of the position of the defective pixel.
In the structure stated above, image data outputted corresponding to the defective pixel are corrected based on information of the position of the defective pixel which is updated for each image pickup of the image for detecting pixel defects, and abnormal image data caused by the defective pixel are compensated.
Item 20
The aforesaid correction is made by replacing image data of a pixel adjoining the detected defective pixel with image data of the defective pixel.
In the structure stated above, image data which are close to image data corresponding to the defective pixel are frequently obtained in a normal pixel that adjoins the defective pixel. Therefore, image data of the defective pixel are not used as they are, but they are replaced with image data of the aforesaid adjoining pixel.
Item 21
The solid image pickup element mentioned above is provided with plural filters each being of a different color, and the aforesaid correction is made by replacing image data of the defective pixel with a mean value of image data of peripheral pixels each being provided with a color filter identical to that for the detected defective pixel.
In the structure stated above, a mean value of the pixels surrounding the defective pixel approximates stably to image data corresponding to the defective pixel. Therefore, image data of the defective pixel are replaced with the mean value.
Item 22
There is provided a mode selecting means which selects between a mode for image pickup of the image for detecting pixel defects and a mode for detection of a defective pixel.
In the structure stated above, when a mode to detect the defective pixel is designated through the mode selection conducted by the mode selecting means, detection of the defective pixel is carried out by regarding the image obtained after image pickup as an image for detecting pixel defects.
Item 23
An electronic camera equipped with a solid image pickup element wherein there are provided a control means which controls the solid image pickup element so that an image for detecting pixel defects may be photographed by one image plane, a pixel defect detecting means which detects pixel defects by comparing a signal level of electric signals for one image plane outputted from the solid image pickup element with a prescribed threshold value for each pixel, a rewritable storing means which stores information of the position for detection of pixel defects, and a pixel defect correcting means which corrects the signal level of the position for detection of the pixel defects to the prescribed level.
In the structure stated above, the solid image pickup element controlled by the control means, and an image for detecting pixel defects is photographed by one image plane. The signal level of electric signals for each one image plane outputted from the solid image pickup element is compared with the prescribed threshold value for each pixel by the pixel defect detecting means. When there are pixel defects, the pixel defects are detected by comparing the signal level with the prescribed threshold value for each pixel, because the signal level for the pixel defects is greatly different from the signal level of other pixels when they are compared. When the pixel defects are detected, information of the position for detection of the pixel defects is stored in the rewritable storing means. Then, based on this positional information, the pixel defect correcting means corrects the signal level of the position for detection of the pixel defects to the prescribed level. The storing means which is rewritable and the pixel defect detecting means which is provided as stated above make it possible to detect pixel defects not only at the time of factory shipments but also at any time.
Item 24
While a light quantity adjusting means which adjusts light quantity of the solid image pickup element is provided, the control means controls the light quantity adjusting means so that light quantity may be zero when detecting white flaws as pixel defects, and the pixel defect detecting means compares a signal level of electric signals outputted from the solid image pickup element with the prescribed threshold value for detecting white flaws.
In the structure stated above, when detecting white flaws as a defective pixel, the light quantity adjusting means is controlled by the control means, and light quantity of the solid image pickup element is adjusted so that it may become zero. Then the electric signal of the solid image pickup element is compared with the prescribed threshold value for detecting white flaws, whereby the white flaws are detected.
Item 25
An electronic camera is provided with a solid image pickup element having plural pixels, a storing means which stores information of defective pixels of the solid image pickup element corresponding to exposure time, and a determining means which determines defective pixels based on information of the exposure time in the course of photographing and information of the defective pixels.
Item 26
An electronic camera is provided with a solid image pickup element having plural pixels, a storing means which stores information of defective pixels of the solid image pickup element corresponding to temperature of the solid image pickup element, and a determining means which determines defective pixels based on information of the temperature of the solid image pickup element in the course of photographing and information of the defective pixels.
Item 27
A pixel signal correcting method includes therein a step to determine defective pixels in the course of image pickup based on both information of defective pixels corresponding to the exposure time relating to a solid image pickup element having therein plural pixels and information of exposure time in the course of image pickup, and a step to correct the signals outputted from the determined defective pixels.
Item 28
A pixel signal correcting method includes therein a step to determine defective pixels in the course of image pickup based on both information of defective pixels corresponding to the exposure time relating to a solid image pickup element having therein plural pixels and information of temperature of the solid image pickup element in the course of image pickup, and a step to correct the output signals from the determined defective pixels.
Item 29
A pixel signal correcting method includes therein a step to obtain information of defective pixels corresponding to each exposure time by conducting image pickup with a solid image pickup element having therein plural pixels while changing exposure time, a step to determine defective pixels in the course of image pickup based on both information of exposure time in the course of image pickup and information of the defective pixels determined, and a step to correct the output signals from the defective pixels determined.
The structure described in Item 1 makes it possible to detect defective pixels which are caused after changes with time, because defective pixels (white flaws and black flaws) of the solid image pickup element can be detected based on results of photographing with an electronic camera.
The structure described in Item 2 makes it possible to detect, through a simple structure, the black flaws and white flaws which output signals extremely different from output of normal pixels.
The structure described in Item 3 makes it possible to detect defective pixels without being affected by dispersion of images, by discriminating image data using the standard corresponding to the image obtained through photographing.
The structure described in Item 4 makes it possible to detect defective pixels without being affected by a difference of luminance caused by a difference of a color, in an electronic camera wherein plural color filters each being of a different color are provided on a solid image pickup element for color photographing.
The structure described in Item 5 makes it possible to judge accurately whether pixels are defective or not with image data within an area where the targeted pixel is centered.
The structure described in Item 6 makes it unnecessary to calculate the reference image data for each targeted pixel and thereby to lighten a burden of calculation.
The structure described in Item 7 makes it possible to judge defective pixels through a difference of output levels while taking the level of image data of surrounding pixels into consideration.
The structure described in Item 8 makes it possible to detect defective pixels simply based on a difference of output levels caused by pixel defects.
The structure described in Item 9 makes it possible to lighten a burden of calculation by making calculation of reference image data to be unnecessary for each targeted pixel, and thereby to finally judge accurately whether a pixel is defective or not, using the reference of image data in a pixel area where the targeted pixel is centered.
The structure described in Item 10 makes it possible to avoid that defective pixels are affected by flaws and shades of a subject and thereby are detected erroneously.
The structure described in Item 11 makes it possible to detect defective pixels from results of photographing without limiting a subject in detail.
The structure described in Item 12 makes it possible to secure a contrast between image data of defective pixels and peripheral pixels without being affected by dispersion of subjects and thereby to improve detection accuracy.
The structures described in Items 13 and 14 make it possible to avoid that defective pixels are affected by noise and thereby are detected erroneously.
The structure described in Item 15 makes it possible to detect white flaws simply and accurately through photographing of a dark image.
The structure described in Item 16 makes it possible to detect white flaws automatically each time the power supply is turned on, and thereby to detect surely the state of the white flaws before photographing.
The structure described in Item 17 makes it possible to detect changes of increase in white flaws caused by temperature rise of the solid image pickup element.
In the structure described in Item 18, detection of white flaws is conducted at prescribed intervals and thereby it is possible to learn clearly changes with time for the white flaws.
The structure described in Item 19 makes it possible to avoid that image quality of a photographed image is lowered sharply by existence of defective pixels.
The structures described in Items 20 and 21 make it possible to correct to the value which is close to the data which should be obtained originally as image data of a defective pixel.
The structure described in Item 22 makes it possible to conduct detection of defective pixels at any time.
The structure described in Item 23 makes it possible to conduct correction of pixel defects corresponding to changes with time of the solid image pickup element, thereby to utilize even an image pickup element having pixel defects slightly, and thereby to achieve cost reduction.
The structure described in Item 24 makes it possible to avoid deterioration of image quality caused by white flaws by detecting white flaws highly accurately and by correcting image data for the portion of the detected white flaws.
The electronic camera with the structure described in Item 25 having therein a solid image pickup element having plural pixels, a storing means which stores information of defective pixels of the solid image pickup element corresponding to exposure time and a determining means which determines defective pixels based on information of exposure time in photographing and information of the defective pixels, makes it possible reduce the number of defective pixels determined by the determining means when the exposure time is short, for example, and thereby to conduct quickly correction processing for output signals. On the other hand, when the exposure time is long, it is possible to enhance the number of defective pixels determined by the determining means to the necessary level, and thereby to improve image quality.
The electronic camera with the structure described in Item 26 having therein a solid image pickup element having plural pixels, a storing means which stores information of defective pixels of the solid image pickup element corresponding to temperature of the solid image pickup element and a determining means which determines defective pixels based on information of temperature of the solid image pickup element in photographing and information of the defective pixels, makes it possible reduce the number of defective pixels determined by the determining means when the temperature of the solid image pickup element is low, and thereby to conduct quickly correction processing for output signals. On the other hand, when the temperature of the solid image pickup element is high, it is possible to enhance the number of defective pixels determined by the determining means to the necessary level, and thereby to improve image quality.
A pixel signal correcting method described in Item 27 having therein a step to determine defective pixels in the course of image pickup based on both information of defective pixels corresponding to the exposure time relating to a solid image pickup element having therein plural pixels and information of exposure time in the course of image pickup, and a step to correct the output signals from the determined defective pixels, makes it possible to reduce the number of defective pixels when the exposure time is short, for example, and thereby to conduct quickly correction processing for output signals. When the exposure time is long, on the other hand, it is possible to enhance the number of defective pixels to the necessary level, and thereby to improve image quality.
A pixel signal correcting method described in Item 27 having therein a step to determine defective pixels in the course of image pickup based on both information of defective pixels corresponding to the exposure time relating to a solid image pickup element having therein plural pixels and information of temperature of the solid image pickup element in the course of image pickup, and a step to correct the output signals from the determined defective pixels, makes it possible to reduce the number of defective pixels when the temperature of the solid image pickup element is low, for example, and thereby to conduct quickly correction processing for output signals. When the temperature of the solid image pickup element is high, on the other hand, it is possible to enhance the number of defective pixels to the necessary level, and thereby to improve image quality.
A pixel signal correcting method described in Item 29 having therein a step to obtain information of defective pixels corresponding to each exposure time by photographing while changing exposure time with a solid image pickup element having plural pixels, a step to determine defective pixels in the course of image pickup based on information of exposure time in the course of image pickup and information of the obtained defective pixels, and a step to correct the output signals from the determined defective pixels, makes it possible, when the number of defective pixels varies depending on the length of exposure time, to improve image quality while securing quick correction processing of output signals by increasing or decreasing the number of defective pixels whose output signals are to be corrected, in accordance with exposure time.